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ABSTRACT 

The course units which form the core of the four 
foundation courses at the Open University were sampled for 
readability, using an electronic version of R. Flesch’s (1951) 

Reading Ease Score. The textload (reading workload) of the courses 
was predicted by estimating the total length in words of all text 
components of these cour s es . Large var i at i ons in readab i 1 i ty occurred 
between authors, topics, and courses. Large variations also occurred 
in the overall textload on these courses. The single best predictor 
of the reading difficulty of a course unit is the identity of the 
author, which suggests that one route to better readability would lie 
in the improved management of human resources in a course team. A 
substantial amount of the prose was more difficult than passage 3 on 
the original cloze reading test: prose difficulty is therefore likely 
to be a major factor in hindering course completion for students with 
weaker reading skills. (Contains 16 references, and 7 charts and 
figures of data. Four appendixes present data for each of the 4 
foundation courses.) (Author/RS) 
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The course units which form the core of four foundation courses at the Open 
University were sampled for readability, using an electronic version of Flesch's 
Reading Ease Score. The textload (reading workload) of the courses was 
predicted by estimating the total length m words of all the text components of 
these courses. Large variations in readability occur between authors, topics, 
and courses. Large variations also occur in the overall textload on these courses. 
The single best predictor of the reading difficulty of a course unit is the identity 
of the author, which suggests that one route to better readability would lie in the 
improved management of human resources in a course team. 

A substantial amount ofthe prose is more difficult thanpassage 3 on our original 
cloze reading test: prose difficulty is therefore likely to be a major factor in 
hindering course completion for smdents with weaker reading skills. 






Introduction 

The work reported here is part of an investigation 
into the reading difficulties faced by students 
entering the OU at foundation course level. This 
project has two main parts. The first part, an 
investigation of OU students' reading skfils. has 
been the subject of three previous reports 
(Macdonald-Ross & Scott 1995a. b: Scctt & 
Macdonald -Ross 1995). This report deals with 
the readabfiity of OU foundation course texts. 

The total text length and readabiliry of four 
foundation courses was estimated. It is now clear 
that much foundation course material is 
linguistically difficult for many ofthe students. An 
analysis of the third level history course A3 17 
revealed a similar pattern. 

The work involves certain novel features: it is 
one of the largest surveys of British educational 
learning material, and probably the largest 
survey of the reading difficulty of higher 
educational material in any country. The 
existence of printed teaching material, and 
particularly text in electronic form . made possible 
a much more extensive samphng than would 
have been possible by hand. Lastly, the data can 
be related to the known reading capaciry of the 
audience, as measured earlier by our reading 
tests. This has rarely (if ever) been achieved for 
such a large group of learners m higher 
education. 

Michael Macdonald-Ross 
Bernard Scott 
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The rationale 

One thing that distinguishes this research from 
most other research projects on our students is 
its concentration on psychological aspects of 
student learning. We are attempting to test a 
model of the effect of reading mismatch on 
student learning and progress. 

In our model (figure 1) we suppose that 
mismatch between the reading skilis of the 
students and the readability of the course 
materials is an important cause of student drop- 
out and failure. We compare our data on student 
reading skills and text readability levels to data 
on the. progress of our cohort L.’-oughout the 
year. So far the model has worked well. 

reading skits readability of 

01 students course materials 




negative eaect on study 



3$ bv 

effect on student progress 

Figure 1 . A model of the central hypothesis 
A facet not captured by this graphic is the vanety of 
other factors which also impact on student progress. 
The standmg of reading mismatch relative to the other 
known factors in student progress is being mveshgaiad 
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Readability measures 

Readability measures of the h/pe v/e used are 
mdices that predict: they do not explain how 
students learn, or teach authors how to v/nte. 
They do not measure the conceptual difficulr/ of 
the subject-matter, unless indirectly thus shows 
Itself in more difficult prose. 

V'.liat they do - and do well - is to predict the 
difficulty of prose, especially if there is some 
information about the reading skills of the readers. 

In this report, then, the readability scores quoted 
are measures of syntactic and semantic 
ccmplexitv' that predict the difficulty of text 
comprehension. This operational definition does 
not deny that many other factors make up a text, 
nor that mvestigation into leamability . concepmal 
comprehension, prose style and the effect of 
broader study skills may be relevant to a full 
appreciation of reader-text interaction. 

Readability measures have been cnticised. 
usually by authors who fail to discriminate 
between an index and an explanation, or between 
the production of prose and its predicted effect 
on the reader. 

Our view on the matter is that, providing their 
limitations are well understood, there are few 
metrics in the whole of the social sciences which 
have been as extensively validated as the 
readability measures w'e use: their effectiveness 
at prediction has been well demonstrated. 

Most formulae enshrine a balance between 
ease of application and coverage of linguistic 
variables. Thus, good prediction may be got by 
two-variable formulae where one vanabie is an 
mdex of semantic complexity (usually a direct or 
indirect measure of the vocabulary demands of 
the text), and the other vanabie is an mdex of 
syntactic complexity (usually sentence length). 

These two indices combined are highly 
predictive of prose difficulty as expenenced by 
readers. "The good news is that t.vo language 
variables, and simple ones at that, account for so 
much of the vanance in the prediction of readable 
•OTitmg" (Klare 1984). More complex formulae 
developed for special purposes (such as 
linguistic resectrch) add little extra to the accuracy 
of prediction. 

Because of the extent of the literature on 
readability, it is not a simple matter to give an 
overview of the field. There are hundreds of 
studies on the effect of readability on vanous age 
crroups. and of its effect under vanous conditions 
of motivation, pnor knowledge of subject-matter, 
and Its relation to the readmg slolls of subjects. 
The mam resources to gam access to this 



literature are the reviev/s by George Hare ( 1 963, 
1974 and 1984). The topic was also addressed 
by Macdonald-Ross ( 1 979) , and by Hirsch in his 
Philosophy 01 Composition (1977). 

A few examples; Klare and Smart ( 1 973) found 
a close relationship between readability level of 
correspondence course material and the 
probability- that correspondence students would 
persevere and complete them courses. Murphy 
(1947a,b) and Swanson (1948) found that 
readership of newspapers goes up with 
improved readability in split-run studies of 
newspapers. Sticht and colleagues showed that 
readability had a significant effect on the use of 
military manuals (Kem, Sticht & Fox 1970), and 
Johnson et al (1972) showed the effect of 
readability on discrepancies m folio wmg military 
procedures. 

These results suggest that readability 
measures are as applicable to adults as to 
younger age groups. Indeed, the two best- 
knowm formulae, the Flesch Reading Ease Score 
and the Dale-Chall formula, were actually 
developed for adult matenals and validated 
against standard criteria for adult readers. 

Textload 

We use the term textload for that part of the 
coursework which is textual. The readability of 
texts has an effect on study which varies 
according to the reading skills of the students. 
The magnitude of this effect is influenced by the 
quantity of text and whether there is time in the 
schedule for re-readinganddecodingmeanmg. 

It has been shown by Klare in a series of studies 
that the effect of a mismatch between reading 
skills and text readability is more pronounced if 
subjects are put under tune pressure: learner 
motivation and familianty and interest m the 
specific subject-matter are other relevant factors. 

Assummg learner motivation i6 fcurly high 
throughout our foundation courses (there is much 
mformal evadence to suppose so), variation in 
course length remains as a significant factor. 
The reason for this is obvious: the heavier the 
workload, the more the student is under pressure 
to keep up wth the work. The most significant 
element in course workload is the textual 
component. Our students are adults, often with 
jobs and families, which means that the pressure 
of workload is real, and may become cnncal. 

Our system of contmuous assessment means 
that students are dnven by a schedule which 
cannot be easily evaded if they wish to succeed 
on the course. Tactics such as selective readmg 
are no doubt often used (here we suffer from a 
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lack of good diary or observational data) but 
again the nature of cur courses means that there 
IS not so much incidental material as there might 
be in a traditional umversity course. 

We v/ould expea to see a greater effea of 
readability on overloaded courses and a lesser 
effect on underloaded courses (if any of the latter 
exist'.). After its first few years the University has 
made great efforts to keep the students' workload 
within bounds. Courses are expeaed to contain 
about melve hours work per week, and survey 
feedback is used to recover rough data on the 
time taken by students for smdy. There is little 
data on the total quantity of text in v/ords, partly 
because we have only recently held the course 
text electromcally. and partly because each 
course has a large number of different text 
elements. 

We can get easily the total word count for the 
course umt matenai for any course which is held 
elearonically, but aU courses have a vaner; of 
booklets, set books, readers and guides, which 
raise the total word count. To give some idea, 
here are the overall figures for two foundation 
courses [estimated by samplmgwhereelectronic 
text was not available] : 

Social Science D 103: course units 574,000 
other 323,550 

total: 897.550 

ArtsA102; course umts 512,200 

other 663,100 

total: 1,175,300 

A clear pattern emerges here: A 102 is slightly 
shorter than D 103 if only the course umts are 
considered, but the other course text m A102 is 
over tmce as long as the other course text in 
D 1 03. Such a difference is probably significant in 
its impact on student workload. 

According to Ronald Carver's important and 
comprehensive revnew of the speed of reading 
(1990) 'as long as college students and ad’olts 
engage then raudmg process they tend to read 
at a relatively constant rate around 300 words 
per minute even when the materials vary 
drastically in terms of difficulty' level". Carver s 
rauding [reading for mearung] is his third gear. 
Lower gears are for learning and are 
correspondingly slower. Note that most units 
contam Activities and SAO's [self-assessment 
questions] Considermg all this, a mean reading 
speed for comprehension can hardly be more 
than 1 50 wpm. and might well be less. 

Let us dc the calculations for D 103. 

2c weeks x 12 hours = 384 hours study time 

997550/150 = 5983 3 rrunutesor about 1 00 hours 



[99.721 to read text materials once. 

AUovnng 2 hours per tv programme. 30 minutes 
per radio programme and 2 hours for each 1 
hour tape g.ves 54 hours for other media. 384- 
54 = 330 hours. 

The Course Guide says about half of study time 
IS for Course Units (and set book and reader’’) . 
Thiis IS about 190 hours. This leaves 140 hours 
for wnting TMAs [tutor-marked assignments] . 
though It is difficult to separate reading and 
wnting aspects of smdy. For 8 TMA's. this is 
17 5 hours per TMA 'perhaps an under- 
estimate] . 

It looks as if a student who studied 1 2 hours per 
week, of which an average of 6 hours was spent 
reading printed course materials, could hope to 
read everything in D 103 once, and most of it 
twice. That is probably about right, so we 
conclude that D103 is not overloaded (in that 
respect, a triumph of course management). 

By comparison, A102 contains significantly 
more text than D103, due entirely to the non- 
course unit material. If D103 is about right, then 
A102 is almost certainly overloaded, a factor 
which will lead to selective reading by some 
students and expenditure ofextra time by others. 
One can be fairly certain that students with 
borderline reading skill wOl find it a difficult 
course: the pressure on their time will make the 
text readability a critical faaor. 

We clearly need empirical data from 
observational and diary smdies to know how 
students cope with these difficulties. Such data is 
not available at present, and is not by any means 
easy to obtam m a distance learning system: 



textload x foundation course 




foundancn course 
: Q ur.b D c*r.c: 



Figure 2 The textioad of four foundation courses. 
A 102 (*^jTts) has the greatest textload, and T102 
(Technology) the least Since ail courses have various 
non-text actmties. one cannot reach conclusions about 
overall course workload from this data alone 
Note that SI 02 has the most text in the form of course 
units and little other text Neither SI 02 nor T 102 use 
set books or readers on these courses. 



Sampling 

Before the sampling a certain amount of electronic 
text editing was necessary. This was done t: 
remove the non-prose content of the courses 
(notations, diagrams, graphics, tables, and 
complex typographic formats'). Readability 
formula v/ere devised to be used on continuo'us 
prose; and it is obvious that Open Universir; 
course units contain varying amounts of non- 
prose content (in general, all forms of text contam 
more graphic and rypographic elements than 
would have been the case half a century ago). 
Sirice the theoretical basis of the formulae was 
the linguistic structure of continuo'is prose, it is 
obvious that meaningless readings would be 
obtained from the non-prose sections. These 
were therefore edited out on our electronic files 
before sampling began. 

In nesch's instructions for sampling (Flesch 
1951) he advocates taking 25-30 samples of i QQ 
words each for a ' book' . This amounts to a sample 
of about 2° 0 ofthe text. In our case we have taken 
aboutthe same densityofsamples. butthesample 
length averaged over 1 ,000 words. The effect of 
this was to expand our samples to about 20-- d of 
the text (it v/as the availability^ of electronic files 
and the electronic application of the formula that 
made this possible). Therefore, we have every 
reason to believe that our samples v/ere a fair 
and adequate representation of the prose 
element of the texts. 

readability by faculty 
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Figure 3: mean readabuiU- ofthe foundation 
course texts of four faculties. 

The central points are mean of means. For Ai C2 
D 1 03 and S 1 02 this :s the mean of means for units 
and for T1 02 it is the mean ci means for topics Tne 
upper and lower bounds mar nthe 95^ b confidence 
intervals. 

The upper horizontal line t. figure 2 corresponds 
to the readability of our test passage ^2 (on which 
our sample did well) and the lower line 
corresponds to the readability’ of our passage 3 
which 95° 0 of our sample did not do w^elll 



Comparisons between faculties 

There are strikmgdiffererices between the mean 
readability scores of the four faculties (Figure 2). 

If we ask v/hether faculties pitch their prose at 
the appropriate level for their students, then in 
the case of Technology the answer is broadly, 
yes. Also, the Faculty of Science could expect, 
with perhaps a more targeted editing effort, to 
adjust tlieir text until it also matched the students ' 
capabilities. 

The Arts Faculry prose is overall significantly 
more difficult than the Technology faculty. Some 
allowance can be made for the fact that in our 
test sample the mean scores for Arts students 
were the best of the four faculties, and the 
Technology' students the lowest. However these 
differences were not so large as the differences 
in mean readability of the course units. One can 
conclude that the Arts foundation course is not 
only somewhat overloaded, but is demanding 
in the level of reading skill required of students. 

Lastly, we come to the Social Sciences 
foundation course, where the overall-difficulty of 
the prose in the course units v/ill without question 
cause difficulty for many of their students. 
However, as we saw above, the Social Sciences 
faculty have been more successful than Arts m 
controlling the overall text workload on the 
course. That is a counterbalancing factor. 

Variation between course units 

Of course, the faculty means mask great variety 
at the course unit level. Every' faculty has some 
units (or topics) -vvhich are written in entirely 
suitable prose for their students, and every 
faculty has material which is wntten in prose 
which demands reading skills which we now 
know most of our students do not have. 
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Figure 4- Vanation v/ithm faculties at course 
unit or topic level (contmued over). 
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Figure 4 Variation v/ithin faculties at course unit 
or topic level. 

Variation between blocks or disciplines 

Different courses group their units by vanous 
principles; two of which stand out. One can group 
by discipline or quasi-discipline (SI 03, D103, 
.^102) or one can group by physical binding 
('the block'). 

T102 is composed of blocks which have a 
highly complex senes of threads or topics running 
through the.m 

Variation between authors 

We have data on mdividual authors for those 
courses which idenuh' the authors. A striking 
finding is the narrow range of scores for each 
author (as summarised by the standard deviations 
and 95°'b confidence intervals) compared to the 
rangeofmeanscoresfortliewholegroupofauthors. 

This means that to a large extent readability scores 
are characteristic of the individual and remain fairly 
constant even when individuals wnte on different 
topics. 
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Figure 5 Vanauen between blocks or discplines 
The apparently greater vanability within D 1 03 blocks 
compared to the other faculties is an artefc .ct of the 



graphical scaling; the summary data is m Aj* pendix 2 
S102 and D 103 have blcclcs which are discr^lme-basea 
where T 102 has blocks which are theme-l' id 
in T 1 02 the numeracy and computing sc ctions have 
been taken out of blocks and presented separately 
(N+C) The aberrant block 2 is a 1996 replacement 
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A102 readability by authors 
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SD=4.2: 95Vc confidence ir.ter.’ai = 42.4-46 7 



In the A1 02 data the total range of author means 
IS 28 points on the Resch RES. yet the maximum 
standard deviation of any individual author is 
only 3.6. To put it another way. the best predictor 
of the readability of a section of course text is the 
identity of its author. 

Many of the questions one would like to ask 
have not been investigated. For example, we do 
not know much from previous research about 
the extent to which individuals can adapt their 
v/nting style; there is likely to be much variation 
in this also. At any rate, over the production penod 
of a foundation course our authors are remarkably 
consistent m the difficulty of the prose they turn out. 

D103 readability by authors 
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Figure 6 Vanauon in course unit readability 
m secfuence :rom first to last unit in course 



The D103 author data is similar: the range of 
mean RES between authors is 2 1 whereas the 
highest standard deviation of individual authors 
IS 3.7. 
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Variation in units by sequence 

In 1 995 D 1 03 tutor-counsellor Roger Fry earned 
.out a survey of course readability using Flesch 
on small samples of 600 words per block (Fry. 
personal communication). He concluded that 
there was no progression m readability through 
the course, a finding which, with our much more 
extensive sampling, v/e can confirm (unit order 
charts, figure 5. previous page). 

Byway of contrast, there is clearly progression 
in S 1 02 , no doubt as a result of deliberate course 
team policy. Of course, it is ar, open question as 
to whether there ought vo be .such progression 
m reading difficulty. Conceptual difficulty v/iU 
increase as the course proceeds, especially m 
subjects that have an hierarchical structure, but 
ought not the prose m^tain a level which allows 
most of the students to engage m the concepts 
rather than struggle wath the English? 

Mathematics foundation courses 

We did not include M 1 0 1 m our survey because 
in 1 996 its life comes to an end. The Mathematics 
and Computing Faculty are creating a suite of 
three half-credit courses to take its place, and 
we will be estimating the readability of prose on 
the level 1 half-credit MU 120 (Open Mathematics) 
during 1996. 



In general, it is valid to use readability measures 
on elementary maths courses since they do 
contain significant amounts of prose. 

Higher level courses 

We may reasonably expect higher level courses 
to exhibit more difficult prose if for no other 
reason than the general lack of support for the 
beleaguered author. These course teams are 
smaller, so each academic does more writing, 
and the constellation of colleagues, editors and 
educational technologists which surround 
foundation courses are absent or scarce in a 
higher level course. We might also expect more 
variation between courses, remembering our 
key finding that the best predictor of the 
readability of prose is the author's identity. 

At present we have data on one higher level 
course, A3 17 Themes in British and American 
History: a comparative approach, c. 1760-1870 
[the title itself is quite stretching!] . The honzontal 
rule at 40 RES marks the difficulty of passage 3 m 
our readingtest. Most ofthe prose is quite difficult, 
and some of it is extremely difficult and bound to 
cause problems for most students. 



FIssch 

RMdtng 

£as« 

Scor*55 



50 



difficult 



A3 17 Essays: readability 
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A Dfaka 2 



45 



40 
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, , Oonnacrii* 2 
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n O 0 «y 



I £mst«y/£ngiand«r 
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i 
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^O'Oay 2 
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25 
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A Hay 



* KiIIiCK 
i^OSrien 

• Oonnacma i 
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J. £n<)lai>dar 
T Kilhck 
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£nglano«r 
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Focus 

Points r P 1 



FP2 



FP3 



FP4 



FP5 



FP6 



FP7 



FP8 




F:?jre 7. Tlie readability of A3 17 m topic sequence 
together with data points for mdi’/idual authors. Data 
and figure courtesy of Clive Lav/iess. 
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Conclusion 

Xot only do our students have widely varying 
reading skills, but our foundation courses 
contain text which varies greatly in its 
readability. Much of the prose is certain to 
cause difficulty with students who score m the 
bottom half of our reading tests. We already 
know that such students axe more likely than 
average to drop out before final registration, 
and our hypothesis is that the difficulty and 
sheer quantity of prose in the courses is one 
main reason for this. 

Data on readability has not been available 
premously to course teams, so perhaps it is no 
surpnse that the means of course units may 
show a spread of up to thirty pomts on ±e 
Fiesch RES scale. This speaks of an absence of 
monitoring which might be corrected in future. 
Perhaps more surprising is that mdividual 
authors are remarkably consistent in the prose 
they turn out . even when they v/nte on differ ent 
topics and units. It will be interesting to see 
hovr flexible they are when presented v/ith this 
kind of data: one expects the better writers to 
be more flexible than the less expert ones. At 
a course teams 's disposal is a variety of editing 
skills vvhich need targeting to be effective. 

For those who like factual nuggets to take 
away and treasure, it is clear that the single 
best predictor of the readability of course 
prose is the identity of its author. 

Although we have collective responsibility 
for our courses, wnriting is a personal activiW. If 
‘he prose is difficult then there is at present no 
formal evidence that later processes improve 
it. Yet a great deal of work goes into the 
discussion, revision and edking of the course 
texts. It can be argued that the prose would be 
in much worse shape if not for the efforts of 
oditiors and others, and no doubt there is 
some truth in that. 

The story of this project is not yet complete 
We now have the data on assignment scores, 
and the fmal examination results for the 1995 
foundation courses. The next report v/iU analyse 
this data, and discuss its relation to our model. 

Postscript 

The reaaabiiiw of the prose in this report is Fiesch P£S -16 
I: calculate this value te>h other than continuous prose 
such as graphics, tables, lists, headings, capticns and 
roterer.cesi was removed This left about J.250 v.'crds o. 
'on'ir.ucus prose The Fiesch Reading Ease Score '.vas got 
bv using the Grammar tool in Microsoft Word aner the 
rrarrmar routines were disabled in Preferences 
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A102 readability 



Appendix 1: A102 data 



A 102 Textload 



There are 34 weeks; 1 week is se: aside for Easier. 1 
weekis set aside for Suminer School. The OU assumes 
a npicaJ week requires 12 hours of srady 
A1C2 Preparatory Pack (c45.100v/) 

A1C2 has 32 units, each of which is pan of one weeks 
srady. They are multiply authored, there are a total of 1 9 
authors. 

Course Guide (cl 9 pages. cSOOvrpp.) 

Glossary (c20p. c50Cv/pp) 

Chronology (cl 000 w) 

Illustration booklet (119 plates index c3000w) 

Broadcast Notes (cl 20 p. c400wpp; and Cassette Notes 
\c30p. c400wpp;. 

Set books: Best G Mid- Victonan Britain (c320p. c35jv/ppj 
Dick- ns C Hard Times with intrc and notes <’c22Qp. 
c3SJwpp). 

Gombnch EH Art and Illusion (c30Cp. c500 v/pp; 5C"b or 
more of this text needs to be read 
Course Reader: Golby J (ed) Culture and Sccier/ m Britain 
1350-90: a source book of comtemporary wntungs. 
\c330p, c450v/pp; 

/^^signments Booklet (cl2p. c4 OOwpp; 

Supplement ar/ Matenai Booklet vc48p. c4 OO'wpp). 

Hard Times Study Guide (cl Op. c400wpp; 

AICZ Literature Supplement (cl2p. c400wpp) 

Library Guide tcl4p. c4C0wpp). 

A 102 Summer School booklet' students' guide {9p. 
c4C0wpp). 

Summer School Texts for project v;ork fvarous). 

A1C2 Summer School Computing Experience: Students 
Guide (12p. c400wpp). 

A1C2 After the Summer School (5p. c400vvpp). 

A 102 Rationale for the Examination ;3p. c300wpp} and 
A 102 Prepanng for the Examination (6p. c500v,pp): 
specimen examination paper (6p. cSOOwpp) 

32 tv programmes of 50 minutes each. With preparation 
and follow-up work study time is expectea to be about 
2 hours per programme. 

There are 1 6 rado programmes of 20 mmutes each study 
time for each expected to be 30 mins. 

5 audio -cassettes (4 at 90 mins, one at 30 mins) 

An estimation of total reading required by number of 
words (Preparatory Pack and Summer School Excusal 
Pack not included). 



Umts 


512.200 


Course Guide 


9.800 


Chronology/ 


1.000 


Glossary" 


10 000 


Broadcast Notes 


50.500 


Cassette Notes 


15.800 


Set books: 




Best 


112.600 


Dickens 


112.000 


Gombnch (50^ d) 


75.000 


Reader Golby (90*^ o') 


137 000 


.Assignments booldet 


4.600 


Supplemental/ 


72 COO 


Hard Times study guide 


4 70C 


Library Guide 


5 500 


SS texts (50"'c) 


31.900 


SS guide 


'• 200 


After SS 


2.000 


Stop Press 


2 500 


Total 


1.175.300 




For each foundation course the readability of over 
100 samples, each of one to r»vo thousand words, is 
the core data. 

This core data is then analys-._d into 'jnit. block, 
discipline, author and unit sequence m the mam 
body of this report. 

A 102: Readability of Course Units 



Unit samples mean RES 


SD 


55rc Cl 


1 


4 


36 


1 .7 


34.9-39.1 


2 


3 


46 


I 4 


43.9-48.1 


3 


4 


44 


' ” 


41.9-46.1 


4-6 


i 


55 


25 


52.0-56.3 


7 


4 


52 


1 7 


49.4-53.6 


8 


3 


58 


1 4 


56.2-60.5 


9 


0 


41 


2 2 


40.0-44 3 


10 


5 


44 


2.0 


42.5-46.7 


11 


4 


47 


■ 7 

*. i 


43.4-47.6 


12 


3 


36 


1.4 


33.9-38.1 


13-15 


12 


50 


33 


48.0-52.2 


16 


3 


45 


1.4 


41.5-45.8 


17 


6 


49 


2.2 


46.9-51.1 


18-19 


8 


37 


26 


34.5-38.8 


20-21 


*7 


47 


2.4 


44.4-48.7 


22 


4 


47 


1 7 


45.9-50.1 


23-24 


5 


38 


2.0 


35.7-39.9 


25 




46 


1.7 


43.1-47.4 


26 


5 


44 


2.2 


42.7-46.9 


27 


3 


50 


1 4 


47.5-51.6 


28 


4 


37 


1 7 


34 6-38.9 


29-30 


1 


39 


2 4 


36.6—40.8 


31-32 


5 


46 


2.0 


43.5-47 7 



Mean of means = 45.3 SD = 5.6 
95% confidence mterval = 43.2-47.4 



A 102: Readability by discipline 



Discipline samples mean RES 


SD 


95%c’ 


History 


18 


41.1 


4 1 


29 0-43 2 


Literature 


13 


49 8 


3.5 


47.7-52.0 


Music 


17 


47 9 


40 


43 8-50 1 


Art History 


12 


42 7 


3.3 


40.6-44 9 


Philosophy 


15 


50 


3.7 


47.9-52.1 


Interdisc. 


41 


42 4 


63 


40.2-44 5 



Mean of means 45 6 SD = 2 2 
95° b confidence mterval = 43 5-47 8 



Key to Units 

.distory 

Literature 

Music 

Philosophy 

Inter disc. 



Units 1-3. 16 28 
Units 4-6. 26 

Units 7-9. 25 Art Historv’ V its lC-12 
Units 13-15 27 
Units 17-24.29-32 



D 103 readability 



Appendix 2: D103 data 
D103 Textload 

Tnere are 34 weeks: 1 v/eek is set aside for Easter. ; 
7 /eekis set aside for Summer Schooi. The CU assumes 
a r/pical week requires 12 hours of study 
El 03 has 32 Units. Each is pan (Course G'uide says 
rcughly half) of one week's study. They are multiply 
authored: 23 authors m toto. Units have sections on 
study skills. There are 8 of these i'3 in E:~ :k 1 1 in 
Block 2, 1 in Block 3. 1 in E; ock 5 1 in Block 5 and 1 in 
Block 7). 

Course Guide (clT pages. c500v\pp; 

Glossary Index fc31 pages. c500 vpp. 

Media Booklet (c76 pages. c350 vpp) 

Set book. The Good Study Guide (not ail has to be 
read; c206 pages. c350 wppj. 

Reader wth 22 chapters to be read. 5oc:en' and Social 
Science: A Reader (c300 pages. c500 vpp) 
Assignments booklets [cll pages. c600 vpp, 
Booklet. After D ICS: OU Courses -Ji Social Science (8 
pages. c750 wppj 

Booklet Preparc:g!crDlQ3SiLTjnerSchzoI (8 pages. 
cdOO wpp) 

Summer School set of workbooks (c65 pages of text. 
c450 wpp) 

For students not attending Summer School there is a 
Simiiner Schooi Excusal Pack (c35 pages. c250 wpp) 
Stop Press messages with nev/s. notes for radio 
programmes and errata (14 pages. c250 wpp) 
DIOS Preparatory Pack for use before starting the 
cours proper. 

15 tv programmes; each is 50 minutes. With 
preparation and follow-up v;ork. study time is 
expected to be about 2 hours per programme. 

1 6 radio programmes; each is 20 minutes, totai study 
time IS expected to be 30 minutes. 

7 audio -cassettes one for each Eloc.<. nach holds a 
maximum of one hour’s listening; extra study time 
IS allocated Audio-cassette 7 is accompanied by a 
booklet. DIOS Endnotes for Block 7 (c 1 5 pages. c300 
^wpp). 

An estimation of total reading required by number of 

v;ords(Preparator^y’ Pack and S'ummer School Excusal 

Pack not included) . 

Umts- 

Course Guide 
Glossary 
Media Bcolclet 
Set Book 
Reader 
TfvlA Bcold^.-ts 
After DIO: 

SS Workbcci3 
Stop Press 
Endnotes 

Total. 




r.escn Readrvg Ease score 



Distr±>ution of D1 03 samples 



D103: readability of course units 



Unit 


# samples 


mean RES SD 


95% Cl 




4 


52 


1.7 


49.6-53.9 


2 


4 


44 


1.7 


42.4-46.5 


3 


4 


47 


1 .7 


43.9-48.1 


4 


3 


40 


1.4 


37.9-42.1 


5 


2 


29 




25.9-30 1 


5 


4 


47 


1.7 


44.6-48.9 




3 


38 


1 4 


36.2—40.5 


3 


4 


47 


1 7 


44.4-48.6 


3 


2 


41 


1 


38.9-43.1 


10 


4 


51 


1 7 


49.1-534 


11 


4 


48 


1.7 


45.1-49.4 


12 


3 


47 


1 4 


45.5-49.8 


13 


2 


41 


1 


38.9-43.1 


14 


3 


35 


1.4 


32.9-37.1 


15 


4 


46 


i.7 


44.1-48.4 


16 


4 


32 


1.7 


30.4-34.6 


17 


4 


28 


1.7 


25.4-29 6 


18 


d 


51 


1 

L 


48.4-52.6 


19 


5 


41 


2 


39.1-43.3 


20 


4 


43 


1.7 


40,1-45.1 


21 


5 


45 


2 


42.8-47.1 


22 


3 


40 


1 4 


36.5-40.8 


23 


4 


45 


1.7 


43.1-47.4 


24 


4 


46 


1 7 


43.1-17.4 


25 


4 


28 


1 7 


26.1-30.4 


26 


4 


46 


1.7 


45.4-19.6 


27 


2 


26 


1 


24.4-28.6 


23 


4 


35 


1 7 


33.1-37 4 


29 


5 


T9 

\Jt-l 


2 


29.3-33.5 


30 


3 


48 


1 4 


45.9-50.1 


31 


4 


38 


1 7 


36.4-40 1 


32 


3 


51 


1 4 


48.2-52.5 


Mean of means = 


: 41.6 CD=5 6 




95'^’cconfidence interval = 


39 4-43.7 




D103: readability of course blocks 


Block samples 


mean 


SD 


95° b Cl 


1 


18 


43 6 


4 1 


41.6-45.8 




14 


43 9 


3 6 


4) .8-46 1 




14 


47 4 


3 6 


45 2-49.5 


4 


13 


37 9 


4 1 


35 8-40 1 


5 


13 


42 4 


4 1 


40 3-44 6 


6 


17 


41 7 


4 0 


39.6-13 8 


7 


'^0 


39 2 


4 6 


25 1-404 


Mean of meariS 


= 42 2 


SD = 24 




95°'. 


c. con*'dence mter\^ai 


= 40.0-44 


3 



574.030 

3 500 
14.500 
25CC0 
72 ICO 

; r, ^ .T 2 
0 C\.'J 

29 250 

^ j ^ ^ 

4 500 

c 9 7. 5 50 words 



•* ^ 



SI 02 readability 



Appendix 3: SI 02 data 
SI 02 Textload 

34 weeks: 1 v/eek is set aside for Easter. 1 week 
IS set aside for Summer School. The OU assumes 
a typical week requires 1 2 hours of study. 

SI 02 has 32 Units. Each is part of one weeks 
study (c. 1500 pages, c. 450 words per page). 
Introduction and Study Guide (c. 30 pages, c 350 
v/ords per page). 

Study Comments (28 pages, c. 250 words per page). 
Course Index [c. 35 pages, c. 600 words per page). 
Tais is prc'/ided for occasional use. for v/hen 
students wish to locate a particular topic ^Aathin the 
course texts. 

Clossar/ (c. 55 pages, c. 300 words per page). 
Assignments Booklets (c. 65 pages, c. 450 words per 
page). 

Inu^ocucuon to Summer School (c. 4 pages, c. 300 words 
per page). 

Summer School Laboratory Notebooks (c 90 pages, 
c. 300 words per page). 

Revision Guide (c. 8 pages, c. 450 words per page). 
Specimen Examination Paper (c. 20 pages, c. 600 
words per page). 

Stop Press messages v/ith news, notes and errata (c. 
10 pages. c.250 words per page). 

35 tv programmes; each is 25 minutes. Notes on the 
content of the programmes are included in the 
relevant Course Unit. 

5 audiO“Cassettes (90 minutes each): total study time 
IS expected to be longer than this. The tapes are 
used with other material in the Course Units to form 
audio-visual sequences, with actmties for the 
student to carry out. 

An Experiment Kit is provided for carrying out 
experiments. Short booklets with the bts give 
instrucuons for settmg up apparatus and provide 
advice on safety. 

Three Computer-Assisted Learning (CAL) 
programmes. 

Preparatory mater u''s. Into Science, for use before 
starting the course proper (c.220 pages, c. 400 
words per page). 

Estimation of total reading required by number 
of words (preparatory matenal. Experiment Kit 
booklets and Course Index not included); 



Umts- 


6750G0 


Introduction and Study Guide 


10500 


Study Comments 


7000 


Glossary 


33000 


Assignments Bocldets 


29200 


Introduction to Summer School 


1200 


Summer School Lab. Notebooks 


27000 


Remsion Guide 


3500 


Specimen Exarranation Paper 


12000 


Stop Press 


2500 


Total 


801000 




Distribution of SI 02 samples 



SI 02: readability of course units 



Unit 4 


samples 


mean RES 


SD 


95% ci 


1 


4 


55 


1.7 


52.9-57.1 


2 


3 


58 


1.4 


56.2-60.5 


3 


3 


57 


1.4 


55.2-59.5 


4 


2 


49 


1.0 


46.9-51.1 


5-6 


8 


48 


2.6 


44.7-49.0 


7-8 


6 


48 


2.2 


46.0-50.3 


9 


3 


48 


1.4 


46.5-50.8 


10 


5 


55 


2.0 


52.5-56.7 


11-12 


5 


50 


2.0 


48.5-52.7 


13-14 


5 


■ 


2.0 


52.3-56.5 


15 


3 


53 


1.4 


50.5-54.8 


17-18 


5 


39 


2.0 


37.9-42.1 


19 


2 


47 


1.0 


44.4-48.6 


20 


2 


49 


1.0 


46.4-50.6 


21 


2 


39 


1.0 


37.4-41.6 


22 


1 


40 


— 


— 


23 


2 


46 


1.0 


44.4-48.6 


24 


3 


48 


1.4 


46.2-50.5 


25 


2 


47 


1.0 


44.4-48.6 


26 


1 


50 


— 


— 


30 


3 


42 


1.4 


40.5-44.8 


31 


'3 


43 


1.4 


40.5-44.8 


32 


4 


37 


1.7 


34.4-38.6 


Mean of means = 


47.9. SD 


= 47 





95% confidence inter/al = 45.8-50.0 



SI 02: readability by discipline 



samples 


m. RES 


SD 


95% ci 


Gen. SCI 


12 


55 


0.0 


53.3-57.5 


Physics 


18 


44 


4.1 


42.0-46.2 


Eanh Sci 


14 


47 


3 5 


45.3-49.6 


Chem 


18 


49 


4.1 


46.9-51.2 


Biology' 


14 


46 


2.6 


43.6-47.8 


Mean of means = 


48 2. SD 







95% conndence inter.'al = 46 1-50 3. 



General Science and Phys' Units 1-4 



Physics: 

Earth Sciences: 

Chemistry: 

Biology- 



Units 9-10. 30-32 
Units 5-8. 27-29 
Units 11-18 
Units 19-26 
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Appendix 4:T102 data 
T 102 Textload 

There are 34 weeks: 1 v/eekis se: aside for Easter. 1 
week is set aside for Summer School . The OU assumes 
a typical week requires i 2 hours of study. 

T102 has 6 Blocks. Each Block one mainstream 
topic and two. three or four tnbutary topics (c993 
pages. cSOOwpp). 

Introduction (c36 pages. cSOOwppj 
In troductory Supplement & Reference Book (c 6 pages . 
c500wpp). 

Supplementary Matenai in each block includes study 
guides, information about assignments, visua. 
materials that accompany audiotapes, selectea 
readings and miscellaneous items of informaucn 
(cl 25 pages. cSOOwpp). 

Software Guide (clO pages. cSOOwpp). This is a 
reference document, to be consulted as required. 
Communications Guide (c30 pages. c300wpp) 
Summer School Handbook (c90 pages. cSOOwpp). 
Specimen Examination Paper (c:5 pages. c500v/pp; 
Preparing for Living Technciogy and associated 

Supplementary Material for use before starting the 
course proper (c70 pages, c500\%tdp). 

Working with Windows, 1 , 2 and 3 to be studied before 
the course begins and for later reference (cl 60 
pages. c300wpp) . The guides are accompanied by 
3 audio-cassettes (90 minutes each) and an 
applications disc. The software supports word- 
processing, spreadsheets and communications. 
There are also Computer Aided Learning (CAL) 
matenals that suppon Numeracy (Blocks 1 , 2 , 3 & 6) 
and Working with Chemical Symbols (Block 4). 

8 tv programmes, each Of 50 minutes. They are 
designed to be studied in conjunction with study 
guides and Block texts. It is recommended that they 
are recorded for repeated viewing and note-taking. 
5 audio-cassettes (90 minutes each); total study "ime 
is expected to be longer than this. The tapes are 
used with other matenai in the Blocks to form audio- 
visual sequences, with acuviues for the student to 
carry out. 

An estimation of total reading required by number of 
words (preparatory m*aterial and Softv/are Guide not 



included): 

Introduction 18.000 

Blocks: 497,500 

Introductory Supplement 3.000 

Supplementary Matenai 62.500 

Communications Guide 9.000 

Summer School Handbeok 45,000 

Specimen Examination Paper 7.500 

Total 642.500 



The most obvious difference from A102 and 
D 1 03 is the absence of Set Bocks or Readers: this 
greatly reduces the reading lead. 



T102: readability of some course topics 



T102 readability 




2:--2S 3:-23 40-49 t-'-Sa ?3-69 TO- 

Hesch Reading Ease score 



Distribution of T 102 samples 



BLOCK 1 








Topic samples 


mean RES 


SD 


95% Cl 


Home 1 0 


51 


3.0 


49.6-53.3 


Heat 5 


59 


2.0 


56.5-60.7 


Str.& Mat. 5 


62 


2.0 


59.7-63.9 


Numeracy 6 


62 


2.2 


59.2-63.5 


Computing 5 


65 


2.0 


63.3-67.5 


BLOCK 2 








Topic samples 


mean RES 


SD 


95% ci 


Comm. 9 


38 


2.8 


36.2-40.5 


Computing 2 


59 


1.0 


56.9-61.1 


BLOCK 3 








Topic samples 


mean RES 


SD 


95% ci 


Electncity 5 


52 


2.0 


50.3-54.5 


Energy 6 


48 


2.2 


46.4-30.6 


BLOCK 4 








Topic samples 


mean RES 


SD 


95% Cl 


Growth 5 


40 


2.0 


33.1-42.3 


Materials 8 


52 


2.6 


49.5-53.7 


Numeracy 4 


66 


1.7 


63.4-67.6 


BLOCK 5 








Topic samples 


mean RES 


SD 


95% ci 


Food 12 


50 


33 


48 5-52.8 


Biology 4 


48 


1.7 


45.9-50 1 


BLOCK 6 








Topic samples 


mean RES 


SD 


OSI-b ci 


Health 9 


47 


2.8 


44.3-48.6 


Biology 4 


40 


1.7 


38.4-42.6 


Numeracy 5 


54 


2.0 


51.9-56.1 



Mean of means of blocks = 52 5. SD - 4.0 
95^-b confidence interval = 50 4-* 54 7 
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Technical Reports 

1 : The revolution in print technology 
2; A postal survey of OU students' reading skills 
3: Results of the survey of OU students’ reading skDls 
4: OU students’ reading skills and final registration 
5: The readability of OU foundation courses 

Newsflash 

1; OU students’ reading skills 
2: Readability of OU courses 



Michael Macdonald-Ross and Bernard Scott 



The readability of OU foundation courses 



From our previous work on reading skills we know 
that many students find academic prose difficult. We 

now have data on thereadabilityoffoundationcourses. 

such data could help courses better match the 
students' reading ability. 

We think that niismatch between the reading skills of 
the students and the readability of course materials 
13 an important cause of student drop-out and failure. 

To test this idea we compare our data on student 
reading skills and text readability levels to data on 
the progress of our cohort throughout the year. So far 
the model has worked weE. 

Of course other factors do affect student progress. 

How reading mismatch stands r elative to other factors 
in student progress is being investigated. 

Readability measures 

Readability measures predict the difficulty of prose; 
usually they combine an index of syntactic difficulty 
(eg sentence length) with an index of vocabulary (eg 
word length) . The most frequent words in a language 
are the best known, andfromthe work of George Tipi 
in the 1930s we know that there is a relationship 

betweenthelengthofwords and their frequency in use. 

For our work we used an electronic version of the 
Flesch Reading Ease Score which is offered as a tool 
in Microsoft Word under the Grammar facility. 

Results 

The chart 'readability by faculty’ (above right) shows 
the mean readability of four foundation courses, and 
illustrates the variation in their mean prose difficulty. 
The upper horizontal line corresponds to the 
readability of our reading test passage 2 (on which 
most students did weU), the lower line corresponc^ 
to the readability of our test passage 3 (on which 9b /o 
of our sample did not do well). 

Faculty of Technology has put in a great effort to 

make T 102 accessible, arid deserves congratulation 
on its readability. The effort was needed because, as 
it happens, technology entrants scored lowest m our 
reading skill tests. 

Each course can be analysed into blocks, units, 
disciplines, authors &c. The samples for each comse 
number over a hundred (eg 122 samples for D103^ 
the length of samples averages weE over a thousand 
words each. This probably makes it the most 
substantial readabEity sample of umversity-level 
teaching material ever conducted. 

Above right, S 1 02 is analysed into disciplines (apart 
from the first data set, the sequence is not in course 
unit order). From this kind of data course team 
management get hints on wMch parts of a course 
might need priority attention during revision and 
editing. 

InD 1 03 the blocks are mostly buEt around disciplines. 
A picture emerges that blocks four and seven have 
significantly more difficult prose than the rest: our 
idea is that in future this kind of data should be picked 
up during course development. 
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Arts Soaal Saence Technology 

Sciences 

MIDI is in its last year, and was not included in this study 
The readability of MU120 wiU be examined later this year. 
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D 1 03 unit readability in order 



A point made by a D103 tutor-counseEor was that the 
course showed no progression in difficulty of prose 
when analysed by unit order. By contrast S 1 02 seems to 
have achieved this quite well. 

S102 readability z umt order 




uru * 




unit # 



D103: Course unit authors in order of readability 



Opposite is a table which shows that ten authors in a 
course produce prose at least as difficult as passage 3 in 
our test; of those ten, six write prose so difficult that it is 
bound to be an obstacle to comprehension and student 
progress. 

The range of means for authors in a course is wide (20 to 
30 points on the Resch RES scale) whereas the range of 
standard deviations for individual authors is quite narrow 
(about 3 points on the scale). Thus the bestpr^ctorofthe 
readabihty of a course unit is the identity of the author. 

A list of authors in order of readability is no comment on 
their quality as expert academics. Of course, readability 
is not a measure of the academic worth of the course 
material; but on the other hand students cannot benefit 
from erudition unless they can ’••'ad and understand it' 
We developed the term 'textload' for the rninimum 
amount of text in words which a student would need to 
read to comply with study guides in a course. The 
variation between faculties is considerable. If D 103 is 
about the right textload (we think it is) then A102 is 
overloaded. This will put extra pressure on students -with 
sub-standard reading skills. 

S 1 02 andTl 02 have less text because they do not use set 
books and readers at foundation course level. Instead, 
they use home experimental kits and computers to allow 
for practical acti-vities associated with the course. 
Therefore, in S 1 02 and T1 02 the textload is only a partial 
guide to the overall workload. Of course, all courses 
have assignments and other activities which compete 
for scarce time. Assignments can be very time- 
consuirung for some students. 

Higher-level courses are likely to have problems, since 
they do not get the attention lavished on foundation 
courses. 

A3 17 is a case in point; notice that much more than half 
the material is more difficult than our passage 3 test. Of 
course, many of the weaker readers vrill have dropped 
out before reachingthirdlevel. This illustratesthe tension 
between the University’s wish to be open, and the 
consequent fact that many students' reading standard is 
closer to the Daily Mail than to our academic prose. 



I AM account ofthis research is 
I available: The readability of 
I OU foundation courses, lET's 
Text i Readers Programme 
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Data+chart 
courtesy of 
Clive Lawless 





n samples mean RES SD 9S% con&dence ml 



Author 


8 


48.5 


2.6 


46.4>S0.6 


Author 2 


7 


47.1 


2.4 


45.0-49.3 


Author 


15 


46.4 


3.7 


44.3-48.5 


Author »4 


5 


46.4 


2 


44.3-48.5 


Author #5 


9 


46.1 


2.8 


44.0-48.2 


Author 


7 


45 


2.4 


42.9-47.1 


Author <>7 


10 


44.2 


3 


42.1-46.3 


Authors 


7 


43.6 


2.4 


41.4-45.7 


Author #10 


3 


40.7 


1.4 


38.5-42.8 


Author #11 


8 


40.2 


2.6 


38.1-42.4 


Author #12 


4 


39.2 


1.7 


37.1-41.4 


Author #13 


3 


38.3 


1.4 


36.2-40.5 


Author #14 


2 


36 


1 


33.9-3ai 


Author #15 


2 


34.5 


1 


32.4-36.6 


Author #16 


2 


34.5 


1 


32.4-36.6 


Author #17 


4 


32.5 


1.7 


30.4-34.6 


/jjtbor #18 


6 


32 


2.2 


29.9-34.1 


Author #19 


2 


27.5 


1 


25.4-29.6 



Mean of means = 39.9. SD = 4.2: 95% confidence interval = 37.7-42.0 
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